DEVONagent Pro logs all pages filtered out in the log, together with a short code describing why the page was filtered. Use the log tab of search windows to find out which pages have been dropped and for what reason.
HTTP Error Codes
These error codes are standard HTTP error codes delivered directly from web server that DEVONagent Pro asked for a file:
-
204: No response
-
301: Moved permanently
-
302: Moved temporarily
-
401: Authorization required
-
403: Forbidden
-
404: Page not found
-
408: Request timeout
-
410: Gone
-
500: Internal server error
-
501: Not implemented
-
502: Service temporarily overloaded
-
503: Gateway timeout
-
522: Connection timed out
Wikipedia has a complete list of all possible HTTP error codes.
DEVONagent Pro Error Descriptions
These errors appear in the log when communication problems with the web server occurred, or, more likely, when DEVONagent Pro filtered the page out due to search strategy or user-defined settings.
-
Advertisements: Link or frame probably contains only advertisement or banners
-
Archived: Identical page has already been archived (if "Archived Pages"filter is activated)
-
Disabled format: Page isn't in a suitable format
-
Disallowed: Page couldn't be downloaded due to permissions
-
Duplicate: Identical page already found
-
Excluded: URL or domain excluded by the user
-
Failed: Search engine could not be reached
-
Filtered by scanner: Page did not pass the selected scanner
-
Host not found: Website not found
-
Identical: Identical page with different URL already found
-
Incomplete: Download could not be completed
-
Junk: Page contains almost no useful information (if the article filter filter is activated)
-
Language: Page does not match the language selected in the language filter
-
Links: x of y: Pages delivered as a result by a search engine. Numbers indicate used links vs. returned links. If the number of used links is lower than the returned one, then either another search engine already returned the same link or the host is blocked.
-
Lookup failure: Domain of website not found
-
No content: Page does not contain any text
-
No match: Page does not match the query term
-
Only links: Only links matching the query term found (if Follow Links is activated)
-
SPAM: Page tried to fool search engines, but does not match the query term without those tricks
-
Similar: Similar page (different layout but similar text) with different URL already found (if Similar Pages is activated)
-
Skipped: Domain was skipped because it matches one or more entries in the list of unwanted sites
-
Timeout: Web server didn't respond in time
-
Too big: Page is larger than the maximum size defined in the preferences
-
Unknown data: No HTML or text content
